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■^fclj^ Abstract 

Three basic ccnceptual issues wVii^h inde.'lie studies ol£ the con- 
tent of instruction are explored In this paper. These include the 
need to: (1) reach a common definition of the concent of contcMit of 
instruction, (2) undertake a serious analysis of the taxonomies presently 
employed in studies of content;, and (?) critically examine the units of 
observation employed. These issues, which cause serious communication 
problems among researchers, are discussed in detail and suggestions for 
their resolution are offered. 
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" * ^ 1 * 

CONCEPTUAL I'SSUES IN THE CONTENT /STRATEGY DISTINCTION 

Donald J. Freeman^^' . , • - i , - 

There is- a growing awareness that studies of the Impact of teaching 
on student performance must take into account content of instruction 
(what is taught) as well as instructional strategies (how something is 
taught). The central role of content analysis in investigations con- 
ducted by Airasion and Mad,aus (1976) , Jenkins' and Pany (1976), 
Armbruster, Steven, and Rosenshlne (1977), Porter, Schmidt, Floden, 
and Freeman (1978), Grosser (1978), Perkins and Buchatian (1978), and 
Buchanan and- Mllazzo -(1978) lib ;trates this trend.. 

Emerging areas of inquiry oft'en generate challenging conceptual 
issues, and increased attention to content of instruction in research 
on teaching is no exception. The major purpose of this paper. is to 
explore three crucial conceptual issues which underlie efforts to study 
the content of Instruction: (1) the need to def in^ the concept x)f con- 
tent; (2) the need to isolate a limited number of acceptable taxonomies 
of content in a given subject matter area; and (3) the need to identify 
an appropriate unit of observation. It should be noted from the outset 
that the intent of the paper is to raise questions suggested by these 
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Indebted t^TTwi^uhs without whose constant assistance and support it 
would n^tfver have finished . 

Donald J. FreemaX is a senior researcher with the Institute for 
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three issues, not to provide answers. It should also be noted that the 
range of professional literature^ dealing with these issues is vast; thus, 
no attempt will be made to provide a comprehensive review of literature 

.1 

in this^ report • . 

The Need to Def Ine Content 
Virtually everyone associated with education has some notion of 
what, is mean^ by "content of 'instruction. " It is clear, however, that 
this meaning varies dramatically from one ^ndi'vidual to tlie next. Thus, 
' it is not surprising that some> of our colleagues within the Institute 
forResearch on Teaching Insist that content of instruction can not be 

V 

differentiated from "instructional strategies"; We feel otherwise. The 
difference between their. stande and .ours is probably due to the fact 
that their concept of content differs from ours. These conceptual 
differences, which cause comryunication problems, suggest the obvious 
need for a clear and generally accepted definition of content which will 
be shared by those conducting research in this' area. 

We have not yet been able to establish a precise definition of 
content. However, we have been able t<^ identify some of the parameters 
which will ultimately characterize our def:tnition. Our deliberations 
suggest that content of instruction: (1) encompasses the full range 
"of intended educational outcomes, (2) is a dichotompus rather than con- 
tlnuous variable, (3) focuses upon intended outcomes rather than actual 
products of ' instruction-, and (A) must be d'eflned from a particular point 
of view or perspective, pur work in the area of content of instruction 
Khas focused on mathematics, but we believe these parameters would apply 
as w.ell to other subject areas. . . ^ 
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Content, as EncompasslnR Full Ran>>e of Intend ed_giilcoine5 

A Prevalent view suggests that content is that body of cognitive 
knowledge which may be passively assimilated by the learner (e.g., Parker 
& Rubin, 1966). We are anxious to avoid narrow boundat;ies such as these 
because they fail to represent adequately alternative points of view which 
might be held among teachers. In our view, content", can include the full 
range of intended educational outcomes which might be successfully assim- 
ilated by the learner. (This concept of content allows for the inclusion 
of such affective outcomes as a positive attitude, toward mathematics and 
such processes as the ability to apply a particular heuristic in problem ' 
solving.) We also believe that content is not implicitly, tied to any 
particular approach to instruction, sucl. as that suggested by the con- 
trast between inductive and deductive strategies. 

Content as a Dlchotomous Variable 

We believe that a precise definition will treat content of in- 
struction as a dichotomous rather than continuous variable. We use the 
terms "content covered" and "content emphasis" to distinguish between 

S 

/these two alternatives. In measuring content Covered, we use a dichoto- 
mous scale; that is, we enter a "1" or "0" in our recordr , depending on 
whether or not a particular content area (e.g., the concept of a tri- 
angle) is covered in a given lesson, textbook, or test. Content em- 
phasis, on the other hand, is measured along some well-defined contin- 
uum. Examples on content emph^-^.sis measures include the amount of time 
spent on a given lesson, the number erf pages in a textbook, and the 
number of items on a test which 'focus on a given topic. Thus, our 
concept of content eniphasis is roughly equivalent to Wiley and 
Harnischfeger's (197A) concept of allocated time and Buchanan and 
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MlUzzo's (1978) concept of conftuU density. ^ 

The need to distinguish between content covered and content ah- 
phasis is not. immediately apparent. Ultimately, however, this differ'entl«■ 
^ tion must be made^to provide'a clour distinction between content (what 
is taught) and instructional strategy (how something is' taught). 

Measures of content emphasis encompass more than simple differ^- 
ences in the extent of content coverage. Variations in time spent on a 
given topic, for example, are 'almos t always accompanied by identifiable 
differences in ins-ructi^onal stra tegies , such as the amount of drill and 
practice which is provided. Time variations arc also linked to dif- 
ferences in student aptitude, the minimal time needed for the student 
to learn what is being taught.- Thus it is impossible to explain pre- 
cisely the influence of content emphasis in observed" relation^ between 
measures of content emphasis and student achievement such as those re- 
ported by Wiley and Harnischf eger (1974). Grosser (1978). and Buchanan . 
ajid Milazza (1978). Any explanation of these relations must include 
at least some reference to differences in other variables, such as in- . • 
structional strategies and student aptitude. When content of instruc- 
tion is treated as a continuous rather than a dichotcmous variable, we 
would have to agree with our IRT colleagues that content and strategy 
caniot be distinguished. ' • 

Co nte n t_a^ Intended ! nstr actional Outcomes 

The third parameter suggested by our eftorts to define the content 
of Instruct^ion is that content must be viewed in terms or intended, 
rather than actual outcomes of instruction. Suppose, for example, that 
a teacher presents a series of problems of the following type^in an 
attempt to enhance £:-udent understanding bt the associative property. 
O . 
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6 + (8 + 3) - • ,' 

a. (6 + 8) -f (6 + 3) 
■ b. (6 + fi) + 9 . . ' ' ■ 

c. (6 + 8) + 3 

Further, imagine that despite this practice and despite further explan-/ 
ations from the teacherf students are still unable to solve the problems 
at the end of the lesson. Under these circumstances, would the content 
of this lesson be the "associative, property" or would there be "no con- 
tent," as the students' performance might suggest? The answer, which 
is clearly suggested by the parameter, is that the lesson deals with 
the "associative property," In other words ."^ content must be definfed 
in terms of what the teacher intends for students to learn, not in terms 
of what they actually learn. What students actually learn is obviously - 
a function of a host of variables (e.g., the appropriateness of the 
instructional strategy, student motivation, and student aptitude). 
Thus, any definition of bontent of instjuctio'n which is phrased in terms 
of actual products rather than intendec' outcomes prec ludesNthe oppor-- 
tunity of drawing a clear distinction between content and instructional 

Strategy. • 

To return to the preceding example, suppose that the students com- 
pletely miss the point of the lesfeon and successfully solv^ each problem 
by simply finding the sura of an alternative which matches the sum of 
the stated number sentence. Under these conditions, would the content 
of the lesson be the "associative property" or "single-digit addition"? 
The answer depends on which perspective one adopts, that of the teacher 
or that of the. student. Thus we arrive at the fourth parameter, the need 
to define content from a particular perspective. 
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Defining Content from a Parti cular Perspective 

We believe that content cannot be defined in some absolute 
9ense. Imagine,, for Instance, that one must determine the content of 
a given classroom lesson. What source of Information should be con- 
sidered? Should the teacher be asked what s/he Intends for students, 
to Learn? Should one examine what actually happens during the presen- 
tation of the lesson, including the teacher's actions and instructional 
materials a student is asked to study? Should students be asked to des- 
cribe what they are trying to learn during' the lesson? Or should the ■ 
post-test associated with the lesson be examined t. determine what 
learning outcomes the test is attempting to measure? 

With the possible exception of the content analysis of the post- 
test, which limits content to those intended outcomes which can be 
meai3ured, each of these sources should provide a valid index o.f the 
lesson content. Yet it. is clear that th? results of the content analyses 
are apt to vary from one perspective to another. What, the teacher in- 
tends for the students to learn, for example, may vary dramatically from 
what the students are trying to learn during the- lesson. Differences 
of this sort r,uggest that reliance on only one source (teachers, stu- 
dents or classroom events) Is likely to result in an incomplete des- 
cription of the content of a given lesson. These differences also 
support the notion that content of instruction mus-t be defln'ed from a 
particular perspective (sUch as that of the teacher or that of the^ 
student) . 

Our research focuses on factors which influence a teacher's selec- 
tion of the content , of instruction; therefore, we are most concerned 
with defining content from the Lc-achfer's perspective. Given this 



7 



perspective and the desiro to treat rontent hi a ce.npr ehens Ive manner, 
we are moving toward a definition wlvlch rnn be represented graphically 
by- the intersection between what the teacher Intends for studentB to 
learn and those classroom ev'ents , (teacher actions and/or instructional 
materials used) which the teacher Judges to be c^^nsisteat with these 
intentions. The teacher's intent^fons might be determined prior, to the 
1 lesson or they may emerge spontaneousb' during the, lesson. ^IC a behav- 
iorlst were to study a teacher "s" intentions , he would probably ask the 
• teacher to indicate what s/he hopes the students Will be able to do 
' following a lesson that they wer.e not able to do before.* The beSiav- 
iorist might also ask what instructional events, if any, occur during 
a lesson presentat ion which are con,slstent with these Intentions. • 

A teacher's intentions may focus on kny type of student learning: 
understanding a concept, developing a m^ore positive self -itnage im- 
proving one's ability to ,.j>ply a particular type of decoding skill, etc. 
Suppose that a teacher .plans and completes a lesson in which the intended 
outcome is increased skil Is in single-digit addition. Suppose . ^too. 
• that embedded within the lesrion is practice with listening and speaking 
skills, appropriate behavior in a peer group, or whatever. AU of these 
activities would be a part of ^he instruc, M.al strategies, but they 
would not be a part of the con^ f the te,. her's single goal fo^r the 
.lesson was single-digit addition. f, on the other hand, listening 
skills were a pnrt of the teacher's gonls f.r the lesson, then the in- 
struction provided in that area would nl so . ■ -nstltute part of the lesson 
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Defining content from a teacher's perspective is most' consistent 



' *This description closely parallels Carroll's (1963) concept of 
a "task." 
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with our basic dcs Ir. t:o' Htudy inclorr, whl.h InllucMKe a toacher's hv\cc- 
tlon of content.' Wo r.co^.Uzv, howcvc . that It is posslblo to fonnulat. 
'a definition from .some other perspective, .uch as that of Htuck■r^t pnr- ^ 
enta, or principals, or from sou.e broader perspective, such as the general 
goals of HchoollnR.. 

r 

Summary 

we are ^ot yet able to provide a prrcise def IniMon of the" concept 
of content 6f instruction, but we have formed several beliefs and iden- 
tified parameters concerning this concept, namely, that thif. concept 
(1) encompasses the full ran^^e of intended ' educat lona 1 outcomes, (2) _ 
should be treated as a dirhotomous rather than a continuous variable, 
(3). should focus upon Intended outcomes rather than actual produe.s of 
instruction, and (4) must be defined from n particular point cf view or 
perspective. To the extent that this paper serves its intended purpose, 
the feedbark we receive slun.ld facilitate refinements in our concept and 
move us toward a clear and generally accepted definition of content 
■ which rair.ht be sharecT by others condxictinR research in this area. 

Td^ntlfvj_nfi Hevelopina Useful Taxonomies 

AssumlpR that it is possible- to develop a standard . definit ion of 
content' of Instruction, the researcher mus-t confront the question of. 
>.t taxonomy should be used to classify the content of a given subject 
^tter area. This, question appears to have a simple answer. The 
mber of --ays in which nbiects or events may be grouped is limitless, 
so all taxonomies are arbitrary. Thus it seems reasonkble to assume 
th^t researchers should develop taxon.^^es wbich best fit their unique 
purposes. Indeed, in many cases, that is exactly what has happened. 
With the^subject matter area of elementary mathematics , *f or example, at 
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least 12 different taxonomies are reported in the literature (see the 
reference section of this paper). These taxonomies have been designed 
t.o servo, a variety of purposes. Including the classification of text- 
books, test development, Instructional planning, measuring the content 
validity of testd, and program eviluation. 

To this author's knowledge, no two researchers have ever used the 
r.ame taxonomy, despite th- fact that they may have shared a common pur- 
pose. Our research group within the TRT, for instance, has designed a 
taxonomy of mathematics content to use in determining similarities and 
differences in the content covered by standardized tests and textbooks, 
while researchers at the Southwest Regional Laboratory have developed 
a different taxonomy to serve roughly the same purpose. Because the 
two groups have used different ':axonomies, the opportunity for compar- 
isons of findings has been severely limited. 

Communication problems of this type lie at the heart of the tax- 
onomy selection issue. If researchers with common goals continue to 
develop and ^dopt different taxonomies, it is unlikely that a consistent 
and comprehensive body of literature regarding content of Instruction 
^111 ever emerge Thus it would be desirable to identify a limited 
number of acceptable taxonomies for a given subject matter area; the 
smaller the number, the greater the ease of communication- Ideally, 
every researcher ^vould use a single taxonomy. 

An F. xample from I V'ol.o^. 

The ideal of using one taxonomy has been realized in the field of 
biology, which has a history of taxonomic development dating back to 
the ancient Greeks. 
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Use of the seven basic Linnean levels is required by 
c onvention , that is, no animal is considered to be satis- 
factorily classified unless it has been placed implic:^tly 
or explicitly in some definite group at each of the seven 
levels. (Simpson, 1961, p. 18. Emphasis supplied by author.) 

Consider the advantage of this convention. Although the purposes of 
a given biologist might be better served by some other taxonomy, such 
as ecological classification (e.g.. saltwater fishes), it is obvious 
that applying one form of classification to all organisms greatly en- 
hances communication among all biologists. 

It may not be possible to develop a single taxonomy which would 
be used hy everyone who studies the content of a given subject matter 
area, but it is clearly desirable to identify a limited number of con- 
tent taxonomies which might be used. Some might argue that this limi- 
tation will occur naturally as research progresses in this area, that 
the better taxonomies will be widely adopted, while the weaker ones 
will disappear. We would counter, however, that researchers should be 
able to increase the rate of taxonomic evolution by identifying the 
qualities of strong taxonomies and supporting those taxonomic develop- 
ments which incorporate these characteristics. Although our deliber- 
ations in this regard have not yet reached the definitive stage, two 
ideas have emerged which seem promising. Assuming that those taxonomies 
which serve the widest range 6f purposes should prevail, then taxonomies 
with certain conceptual bases seem more promising than others. Further, 
these taxonomies which are hierarchically structured seem more functional 
than those which are limited to one level of generality. 

Selecting the Most Useful Taxonomy 

Let us examine the logical base of the criterion we have adopted 
for selecting taxonomies. This b^se is described in straightforward 
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terms by Simpson (1961). 

1. A major function of classification Is to construct 
classes. . .about which we can make generalizations. 

2. The classes are constructed in connection with a 
particular purpose, ^^h±ch depends on the kinds of 
generalizations that are considered pertinent. 

3. Some classifications pert§in to a wider range of 
inductions or to more meaningful generalizations 
than others and are in that sense "better", or 
more useful. (p. 25) 

In greatly simplified terms, xm.agine that taxonomy A adequately 
serves purpose A, while taxonomy B adequately serves both purpose A 
and purpose B. Is it not reasonable to conclude that taxonomy B is 
"better" or "more useful" than taxonomy A? This conclusion might not 
logically follow for someone who Is interested solely in purpose A; it 
does follow if one adopts the more general perspective suggested by the 
need to limit the number of acceptable taxonomies. Viewed from the more 
general perspective, the adoption of taxonomy B should not only satisfy 
the needs of those interested only in purpose A. but should also en- 
hance communication between this group and those who are concerned pri- 
marily with purpose B. If this "which-is-more-usef ul" criterion were used 
to judge the relative merits of alternative taxonomies, the conceptual 
base which undergirds some taxonomies would appear more promising than 
Chat which supports others. • 

At their most .basic level, taxonomies are used to identify members 
of a given class. Taxonomies based on nominal scales serve this pur- 
pose as well as any others. Thus, a librarian concerned with identify- 
• ing and labelin- individual hooks so that they might be stored and 
readily retrTeved is served as well by a nominal system of taxonomy 
as by any other. Likewise, we discovered that a nominal system best 
suited our efforts to determine the content covered in commonly-used 
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standardized tests and mathematics textbooks so we deliberately de- 
veloped a taxonomy of mathematics content based upon a nominal scale. 
(Our principal problem was to determine an appropriate level of gen- 
erality for our categories, not to design a more sophisticated tax- 
onomy . ) 

At a somewhat more advanced level, taxonomies may be used to 
Identify and order members of different classes. Taxonomies of this 
sort are implicitly based upon an ordinal scale. Bloom's (1956) tax- 
onomy, for example, describes a logical order among classes of cogni- 
tive tasks which demand Increasingly complex mental processes. This 
kind of taxonomy is well-suited to identifying general differences in 
the levels of cognitive functioning demanded by a given test or text- 
book, and four of the 12 taxonomies of mathematirs content are based 

in part on Bloom's taxonomy. 

The taxonomy of learning propbsed by Gagn/ (1977) is also ordin- 
■ ally based, with learning at one level represented as a prerequisite 
to learning at a higher level. The main appeal of this taxonomy is 
that it seems to suggest an optimal sequence of instruction, namely, 
to begin'at the lower levels and' systematically progress to the higher 
ones (e.g., teach concepts before rules). If it were possible to prove 
empirically that this sequence is, in fact, optimal (using a research 
strategy similar to that suggested by Eisenberg and Walbesser, 1971), 
the general utility of Gagne'''s taxonomy would be greatly enhanced. Un- 
fortunately, however, it does not seem possible to provide convincing 
evidence, and Gagne-^'s position has been strongly challenged by other 
respected educators such as Bruner (see Shulman, 1970). 

Another limitation of both Bloom's and Gagne''s taxonomies is that 
Er|c they are not particularly well suited to identifying specific similarities 



13 



and differences in content covered by either tests or textbooks. This 
purpose seems better served by nominally-based taxonomies which have 
more specific categories. Thus, to use the terms of our criteria, 
these taxonomies serve purpose B, but not both purpose A and purpose B. 

At their most advanced level, taxonomies may be used to Identify, ■ 
order, and explain the relations among members of different classes. 
Taxonomies of this kind are based on^ ordinal scales which parallel 
natural orders that have been empirically confirmed. The generally 
accepted taxonomy In biology, for ex.nmple, is based not only upon 
similarities among individuals, but on evolutionary relationships, as 
well; empirically confirmed ideas suggested by evolutionary theory ex- 
plain why the relationships depicted in the taxonomy occur. The same is 
true of the generally accepted taxonomic system in geology, where the 
order set forth parallels empirically-established principles of the 
origin of rocks. The inherent strength of taxonomies of this sort is 
that not only do they account for the more obvious associations sug- 
gested by similarities among members of a given class, but they gen- 
er.,to many empirically verifiable inductions as well. Consider a miner, 
for example, who is armed with a classification system based on physical 
similarities among rocks (e.g., rocks which contain copper). This miner 
is at a disadvantage compared to the miner who uses a classification 
system which parallels the order suggested by the theory of the origin 
of rocks; only the latter miner will be able to infer the likely pre- 
sence or absence of copper-bearing rocks in a given area from his class- 
ification of other rocks in that area which are readily observed. 

The generally accepted taxonomies in biology and geology serve 
general purpose C (explai(ation and Inference) AND general purpose B 
(ordering) AND general purpose A (identification). Thus, as Simpson 
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(1961) asserts, there is a general consensus that classxf i'.;ation by 
evolutionary relationships is the "best" method of classifying animals. 

So far, taxonomies used to classify the content of given subject 
matter areas such as mathematics seem well-suited for purpose A (iden-- ^ 
tification) OR purpose I? (ordering), but not for both. It Is also appar- 
ent that none of these taxonomies approaches the level of development sug- 
gested by purpose C. It is not difficult to imagine, however, how such 
a taxonomy might ultimately evolve. Suppose, for example, that it were 
possible to develop a taxonomy which reflected the optimum order of in- 
struction in a given subject area. Suppose, too, that ideas suggested 
by Piaget's theory of cognitive development had been empirically con- 
firmed. If the Instructional sequence suggested by the taxonomy pat- 
alleled that suggested by confirmed stages of cognitive develcpraent , 
then the theory of cognitive development would explain "why" ihis se- 
quence of instruction was optimal. The taxonomy, at this stage of 
development, would be analogous to those used in biology and geology. 

It is highly unlikely, however, that we will arrive at this stage 
of taxonomic development within this, or perhaps even tl?e next, century. 
More immediate benefits are apt to accrue from serious efforts to 
develop taxonomies which serve a variety of lower-level purposes. It 
should be possible, Cor example, to develop taxonomies which reflect 

some meaningful order among classes and which are detailed enough to 

t 

Identify significant similarities and differences among various content 
sources. 

Hierarchically Structured Taxonomies 

One form of ordering among classes which seems especially promising 
is the conceptual hierarchy. One of the principal reasons that existing 
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taxonomies serve a limited range of .purposes is that, in a]jnost all 
cases, categories are based on single levels of generality. Levels 
range from the very general to the very specific; one iiaxonotny reported 
in the reference section, for example, identifies extremely broad areas, 
such as reading and mathematics, while another has 42 categories for the 
classification of a single-dimension, crossed with two other dimen- _ 
sions with up to 10 categories each (Buchanan, 1976). The reason for 
such vast variations in the level of generality in different taxonomies 
is obvious. The purpose of some researchers is best served by more geu- 
eral categories; the purpose of others by more specific aategories. 

As mentioned earlier, the primary problem we faced in selecting a 
taxonomy for our research was determining an appropriate level of gen- 
erality for our categories. At one extreme, we found taxonomies with 
categories too general to detect significant differences in the content 
covered by the standardized tests we were analyzing. At the other ex- • 
treme, we found taxonomies with categories too specific to suggest 
topics wh h teachers could reasonably be expected to focus on in plan- 
ning instruction. (Ultimately, we decided to develop our own taxonomy, 
with catego ies specific enough to identify significant differences, 
yet general enough to be meaningful to t*iachers.) 

Pevhaps problems of this sort would be alleviated -if categories 
were not based on single levels of generality. Suppose that it were 
possible to develop a content taxonomy which was hierarchically struc- 
tured. Consider, for example, the following hierarchy for one area of 
mathematics content: " 
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Mathematics 

Basic computational skills - understanding concepts - ..'application 



♦ / 



subtraction - multiplication - division - addition 

» . . . - 

subtraction w/o borrowing - subtraction with borro wing 



The hierarchical relationship among the categories in this incomplete 
taxonomy is obvious. Any task which is classified at the lowest level 
in the taxonomy (e.g., subtraction with borrowing) " is automatically class- 
ified in one of the categories at each of the higher levels. Tlie advan- 
taee of a taxonomy of this sort is readily apparent; a researcher can use 
content categories that represent the level of generality which best suits 
his/her purposes and still maintain meaningful communication with others 
who have elected to use different levels of generality. It. seems that 
those who are anxious to develop taxonomies of the content of instruction 
in a particular subject matter area should at least consider, the hier- 
archy concept. 

Summary * - . 

The desire for meaningful communication among researchers studying > 

the contant of instruction in a particular subject matter area seems to 
suggest the need for a limited number bf ' acceptable taxonomies; to the 
extent that re&earchers begin to adopt commort taxonomies, communication 
should be greatly enhanced. It is possible to compare the relative merits of 
alternative taxonomies by determining which one effectively serves the 
widest variety of purposes. 

We should begin to encourage the development of content taxoncn;les 
which describe some meaningful order and which are specific enough to 
be used in identifying significant similarities and differences among 
different content sources. The order among classes suggested by a 
hierarchy seems especially promising in this regard. * 
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Identifying an Approp r iate Unit o f Observation 
Imagine that an investigator has developed a precise; and- acceptable 
definition of content and has selected a widely-used taxonomy which is 
consistent with his/her purposes. S/he would still aot be luUy equipped 
to connuct a meaningful analysis of a given content source .(standardized 
test, set of obj'ectlves, textbooks or classroom Instr action) Another 
critical decision that has to be faced is the selection of an appropriate 

unit of observation. 

The following schematic diagram illustrates 'he general set of pro- 
cedures an investigator could follow in attempting to describe the con- 
tent covered by a given source. Specifically, the diagram portrays the 
general steps we followed in classifying one item on a standardized teat. 



Inits of 
observation 
(individual 
test items) 



A standardized 
test (that which 
is being described) 



Content suggested 
by a given unit 



r 




- subtraction 

- subtraction with 

borrowing 

- ability to read and 

understand numerals 
subtracting columns 
of whole numbers 

- applying subtraction 

logarithms 

- etc. 



One dimension of 
our content 
taxonomy 



- add 

r subtract w/o 
borrowihg 

- subtract with 

borrowing 

- add or subtract 

fraction ^ 

- multiply 



The diagram obviously oversimplifies the classification process. 
It seems to imply a particular order, even though there are dyrtsmic ^ 
interactions among the various components. The diagram does, however, 
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serve to highlight' five critical decisions involved in the classification 
process: (It also serves to show how we dealt with this process in the 
development of our taxonomy ^f mathematics content.) 

1. The investigator must carefully daliii-ate what_he Is trying 
to d escribe . In our classification of standardized-tests, 
for example, we decided to attempt to describe the content 
covered in the entire test; an alternative would have been to 

' describe the content of each subtest. 

2. The investigator must decide how to divide what he is tiy\.g . 

to describe into meaningful subsets which will serve as tJe 

units of observation. This decision posed no particular p.cb- 

lenl in our analysis of standardized tests; the logical units 

■ ' ■ • / 
of observation were individual test. items. 

e ■ 

3. During the actual content analysis, the investigator must infer 

the content which is suggested by a given unit of observation. 
jt~ — 

A partial list of the broad array of content suggested by the 

test item 64* is presented in the diagram. . 

-35.' - • ■ ^ 

4. The investigator must select or develop a taxonomy wljich des- 
cribes meaningful content categories. The primary goal is to 
select a taxonomy which provides an adequate description of the 
content suggested by given units of observation. We developed 
a taxonomy which described mathematics topics in terms, of thre£ 
dimensions of a matrix because we felt that thie level 6f gen- 
erality provided the best descriptions of the^principal intent 
of individual test items, 

' 5. Because the content suggested by a given unit of observation is 
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apt to include a b'road array of general intents (see. example 
provided in diagram), the investigator must establish an 
a priori decision rule to limit the range of content ,s /he will 
consider. During our e.^forts to describe the content of Btan- ' 
dardlzed tests, for examp>, we considered only that content 
suggested by a given item which (a) reflected the primary In- 
tent of the item and (b) could be adequately described by one 
or more categories in our taxonomy. For the item in this ex- 
ample, the decision tale dictated that "subtraction with 
borrowing" was the suggested' content which should be consid- 
ered for the operations dimension of our taxonomy. Thus, a 
tally of "1" was entered in the category with this label. 
A general analysis of these five classification decisions .suggests 
there are substantial differences between the problems inherent in the 
^-assification of the content .of standardized tests or lists of in- ^ 
structional objectives and the problems involved in classifying the con- 
tent of textbooks or classroom instruction. Most of these differences 
stem from the fact that s-tandardlzed tests and lists of objectives con- 
sist of sets of discrete content units, while the content of textbooks 
and classroom instruction is not as neatly organized- and occurs as a 
continuous flow* 

TMnHin p a Taxonomy Which Fits th e Unit of Observation 

The segmental nature of standardized tests and Aists of objectives 
greatfy simplifies the task of developing meaningful subsets to serve 
as units of- observation. The only logical units of observation appear 
to be individual test items, in the case of the exams, or individual object- 
O _ Ives, in the case of the objectives. . An asset of these units is tha^' taken 
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direct function of the level of specificity of the taxonomic categories . 
" An investigator classifying content in terras of an entire subject matter- 
art;.w (e«g*» mathematics, reading, or social studies) Is apt to select 
the entire textbook or the course Itself as the unit of observation. 
An investigator identifying topics in a given textbook, on the other 
hand, will probably select a more specific unit of observation, such a'" 
paragraphs or lessons • 

It is possible for the level of specificity suggested by a tax- 
onom) to demand observational units which are so specific that they 
miss the essence of what is being taught. A lesson which focuses on 
division, for' example, might begin with two or three multiplication 
problems which illustr^it^ the relationship between multiplication and 
division. If the units of observation are too specific, these problems 
might be recorded as multiplication even though the obvious intent of 
the lesson is to teach (ilvlsion. For these and other reasons, we have 
deliberately elected to develop a taxonomy which describes topics, 
and we will search for an observational unit which best reflects this 
level of generality whep we begin our classroom observations. 

Selection of a unit of observation should not, as this discussion 
seems to imply, be based solely on the desire to match taxonomic cate- 
gories and observational units* Our deliberations have indicated that 
at least two other criteria must also be applied. First, the appro- 
priate observation unit should be a direct function of the investiga- 
tor's definition of content; the observation unit that we select for our 
classroom observation studies, for example, will somehow have to repre- 
sent the intersection between a teacher's intentions and the Instruc- 
tional events s/he judges to be consiste with these intentions, 
Second, the observational unit must also reflect the natural organization 
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of cextbdbks-or classroom instruction. Chapters, subchapters, and para- 
graphs, for instance, are consistent vith the natural organization of a 
textbook; pages whlcb, arbitrarily divide the text are not. The example 
about the lesson on division cited^earller illustrates how overly spe- . 
ciflG units of observation might fail to capture the natural organization 
of a lesson, V 

» 

Sumtaary 

" The selection of an appropriate unit of observation in content anal- 
yses of textbooks an<;l classroom instruction Is a complex issue. Some- 
how. %n investigator must seek to Identify a unit of observation which 
takes into account the need to match observational units, and categories 
in the taxonomy, to be consistent with his definition of content, and _ 
to represent the na-tural organization of that which is being described. 

Conclusion 

Communication Js the common theme which underlies the discussion, 
of each of the three issues described in this paper. .Until we begin 
to arrive at a common definition of the concept of content of instruc- 
tion, undertake a serious analysis of the taxonomies we are using, and 
critically examine" the units of observation we employ, it will be ex- 
tremely difficult for those of us concerned with the content of instruc- 
tion to communicate clearly and openly j^ith each other.. The purpose of 
this paper has been to fac^ilitate discussion of these three issues and 

J 

to stimulate more detailed reporting practices. Both aotivities— 
dlscusslon and detailed repor ting-are prerequisites to the successful 
resolution of the basic conceptual issues which underlie our efforts to , 
study 'the content of instruction. 
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